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METHOD AND DEVICE FOR DRIVER INFORMATION AND FOR A REACTION 

WHEN LEAVING THE TRAFFIC LANE 

Background Information 
FIELD OF THE INVENTION 

The present invention relates to a method and a device for 
driver information and a reaction upon departure from the 
lane, or, more accurately, in response to the threatening 
leaving of the lane . 

BACKGROUND INFORMATION 

Warning systems, which warn the driver when he leaves the 
lane, or rather, threatens to leave the lane, are known in the 
relat e d art . — EP 1 074 430 Al, — for exampl e , — shows a procedure^ 
For example, European Patent Application No,. 1 074 430 
discusses a method in which, based on image sensor systems, 
the lane in which the vehicle is moving is ascertained, and 
the driver is warned when the vehicle leaves the lane or 
threatens to do so. However, information on the exact boundary 
conditions, under which such a warning is given, is not 
stated- 

Summary of the — Invention 
SUMMARY OF THE INVENTION 

By the driver's taking into account future corrections of the 
track (path) of the vehicle, in ascertaining whether, because 
of due to a future departure from the lane, he should be 
warned or not, it is possible effectively to avoid false 
warnings and, nevertheless, to maintain the desired 
reliability of the warning in response to the occurrence of 
the corresponding traffic situation. 

This makes it possible to recogni z e may allow recognition of a 
future departure from the lane in a particularly an exact and 
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reliable manner. This fact and the suppression of false 
warnings, increases the operating friendliness, and the 
acceptance of a driver assistance system in which the driver 
is warned when the vehicle leaves the lane or upon future 
5 departure from the lane, or rather, reactions are introduced 
which lead the vehicle back to its normal lane (such as by a 
steering intervention) . 

In a particularly advantageous way, — in response to the 
10 estimation of the future path of the vehicle, future 

corrections of the path by the driver -arre mav be taken into 
consideration in an execution that includes his reaction time. 
This l e ads may lead to « further improvement of accuracy and 
reliability of the system. 

15 

An especially simple — implementation is — that an average value 
of the correction measures of the driver, such as a steering 
angle value or a transverse acceleration value is predefined, 
and -irs mav be taken into consideration in determining the 
20 path. The It is also of advantage that the state of attention 
of the driver « mav be taken into consideration in forming 
the magnitude of such a predefined value . 

It is especially advantageous that, — in In the calculation of 
25 the future path of the vehicle, which is drawn upon to warn 

the driver when he leaves the lane, the state of attention of 
the driver w may be taken into account in such a way manner 
that the behavior of the driver in the past is ascertained, 
and, in response to behavior which leads one to conclude that 
30 the driver is inattentive, the possible future path of the 
vehicle ±b may be calculated using a smaller change in the 
curvature than for a driver that is presumed to be attentive. 
It is particularly advantageous to draw upon th e The steering 
motions or the yaw rate changes in the past may be drawn upon 
35 for estimating the degree of attentiveness of the driver. 
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Small steering motions designate an inattentive driver, and 
more forceful motions an attentive one. 

Further advantag e s result from the following description of 
5 ex e mplary embodiments , — and from the dependent claims . 
Brief Description of th e Drawings 

The pres e nt invention is explained in greater detail b e low, 
with the aid of th e specific embodim e nts represent e d in the 
10 drawings > 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows-; — in this context, a block diagram of a device 
for warning a driver and for a reaction when the vehicle 
threatens to leave the lane. 

15 

Figure 2 shows a sketch of a flow chart in which a first 
procedure method for determining the driver warning is shown. 

Figure 3 makes this clearer in the light of its sketches 3a 
20 shows a sketch of a traffic situation. 

Ficfure 3b shows a sketch of a traffic situation . 

Figure 4 shows a flow chart for a second exemplary embodiment, 

25 

whos e proc e dure is made clearer Figure 5 shows a traffic 
situation in the light of the r e pr e sentation in Figure 5 . 
method of Ficgire 4 . 

30 Figures G & 7 show Ficnire 6 shows an exemplary embodiment in 
which the calculation of the possible future path of the 
vehicle is a function of the degree of attentiveness of the 
driver-; — Figur e G showing and shows the situation for an 
attentive driver-; — and Figure 7 ^ 

35 
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Ficmre 7 shows an exemplary embodiment in which the 
calculation of the possible future path of the vehicle is a 
function of the degree of attentiveness of the driver and 
shows the situation for an inattentive driver. 

5 

Description of th e Exemplary Embodiments 
DETAILED DESCRIPTION 

Figure 1 shows a device used for warning a driver and for a 
reaction in response to future leaving of the lane of the 

10 vehicle. What is shown is a control or evaluation unit 10, 

which has includes an input circuit 12, a microcomputer 14 and 
an output circuit 16 . These elements are interconnected via a 
bus system for mutual data exchange- Input lines from various 
measuring devices, via which measuring signals and measuring 

15 data are transmitted, are supplied to input circuit 12 . A 
first input line 2 0 connects input circuit 12 to an image 
sensor system 22, which picks up the scene in front of the 
vehicle. Corresponding image data are transmitted via input 
line 20. Furthermore,, input lines 24 to 28 are provided which 

20 connect input circuit 12 to measuring devices 30 to 34. These 
measuring devices are, for example, measuring devices for 
measuring the vehicle speed, for recording the steering angle 
and for recording additional operating variables of the 
vehicle, which have meaning in connection with the proc e dur e 

25 method described below. At least one warning device 3 8 is 
activated via output circuit 16 and output line 36, for 
instance, a warning lamp and/or a loudspeaker for an 
acoustical warning or a voice response and/or a display for 
displaying an image, with the aid of which the driver is 

3 0 informed and/or warned before the departure, to be feared, 

from the lane, according to the procedure method shown below. 
In addition, or alternatively, it is provided in some 
exemplary embodiments that one should activate a controlling 
system 42 via output circuit 16 and an output line 40, which 

35 automatically leads back into the lane, for instance by an 
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intervention in the steering of the vehicle, and thus prevents 
leaving the lane. 

According to the proc e dure method described below, a warning 
5 or an intervention in the vehicle movement takes place occurs 
when it is estimated that the driver is no longer able to hold 
the vehicle in the lane using normal corrections. To do this, 
a check is made whether, while considering normal corrections 
of the driver, i.e. a certain steering angle which the driver 

10 normally uses to correct his track, the vehicle crosses the 

edge markings or not. The future track of the vehicle is then 
specifically calculated while taking into consideration these 
corrections by the driver that are to be expected, is compared 
to the detected edge markings, and a warning is emitted if the 

15 future track of the vehicle crosses the edge markings. In the 
determination of the value represented by the correction of 
the track by the driver that is to be expected, in one 
supplementary exemplary embodiment, the state of attentiveness 
of the driver is taken into consideration. In this context, 

20 the value is the greater (this corresponds to a more forceful 
correction) , the more attentive the driver is, and the less, 
the more inattentive the driver is. In this context, the 
attentiveness of the driver is derived, depending on the 
exemplary embodiment, from steering wheel movements, head 

25 posture, accelerator motion, etc. 

In an additional exemplary embodiment, the reaction time of 
the driver is also estimated in connection with the detection 
of whether the vehicle will leave the lane in the future. It 

3 0 is assumed that, at the beginning of a warning, the driver 
will have a certain reaction. It has proven suitable to 
subdivide this reaction into several phases. In a first phase 
(of a duration such as 0.5 sec) no change in the steering 
angle takes p lace occurs (reaction time of the driver to the 

35 warning) ; in a second phase, which borders on the first phase 
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(of a duration such as 0.1 sec), the driver undertakes a 
correction away from the edge marking. He may, for example, 
turn the steering wheel by a certain angle (e.g. 10°) in the 
direction that brings the vehicle back into the lane. 
5 Alternatively to this, a rotation of the wheel is carried out 
performed so that a curve is driven that has a transverse 
acceleration of 2 m/sec^. In a third phase, which borders on 
the second phase, driving continues at an unchanged steering 
angle, until the vehicle moves in the direction of the center 

10 of the lane. Under consideration this behavior pattern, the 
future track for each point in time, which the vehicle will 
travel in case of a warning, is calculated. From this 
variable, the suitable point in time of the warning may then 
be derived, which has to take place occur exactly when the 

15 track, calculated in consideration of the driver's reaction, 
crosses the corresponding lane marking by a desired measure 
(which can may also be negative) . The range of distance in 
which the comparison with the edge marking takes place occurs 
is limited, in this context (for instance, to 1 sec) . These 

20 measures ensure that, in each lane-leaving situation, after a 
warning, the driver gets away from his own lane at most only 
by a certain measure . 

Alternatively, or in supplement of the warning, an 
25 intervention in the vehicle movement takes p lace occurs, e.g. 
in the steering^ by which the vehicle gets back to the track 
again in a lane-leaving situation as described above. 

The above -described procedures methods are implemented using 
3 0 programs which are programmed on microcomputer 14. The latter 
generates the output signals for the warning or vehicle 
intervention, with the aid of the programs and according to 
the input signals supplied. In the light of flow diagrams, two 
pr e f e rr e d sp e cific exemplary embodiments of the procedur e 
35 method represented above are shown below. 
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A first exemplary embodiment is described in the light of the 
flow chart of Figure 2. In this context, the program 
represented is run through at certain points in time- 



5 In first step 100, the sequence of the lane marking (such as 
left and/or right lane edge) is read in. The sequence of the 
lane marking is given, in this context, in one prefer r ed 
exemplary embodiment as a polynomial, such as a third order 
power function. This function is derived from the image data 

10 ascertained by the image sensor system In this context, the 
microcomputer, using object recognition measures, recognizes 
the markings, and calculates the parameters of the power 
function from the recognized markings. In subsequent step 102 
the future track of the vehicle is then calculated. The 

15 representation of this track also takes place occurs using the 
function type that is used for the sequence of the lane 
markings. Accordingly, for example, from the current vehicle 
speed, the current steering angle or the transverse 
acceleration, etc, the parameters of the function are 

20 calculated which calculate the pattern of at least one track 
(track of the left and/or right vehicle edge) of the vehicle. 
An additional variable, that is evaluated for the calculation 
of the parameters, is the predefined correction value that -is 
pr e f e rably may be a function of the attentiveness state, which 

25 is worked in, for example, by the correction of the steering 
angle. In subsequent step 104 the two functions are then 
checked as to whether they intersect or whether the future 
vehicle track lies across the pattern of the edge markings 
beyond a certain measure, or whether the future vehicle track 

30 is approaching the edge markings up to a certain measure. The 
distance range (distance from the vehicle) , to which this 
comparison is limited, is predefined, in this context, (e.g. 1 
sec) , or is predefined by the maximum measuring distance of 
the image sensor system. If no crossing (point of 

35 intersection) is ascertained, one must assume that the vehicle 
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remains in the lane, and the program is ended. If, however, a 
crossing of the two track curves are ascertained, then, 
according to step 106 a warning to the driver takes place 
occurs, using acoustical and/or optical means arrangement . and 
5 in one exemplary embodiment an automatic correction of the 

track of the vehicle, for example, by an intervention in the 
steering of the vehicle. The program is then ended and run 
through again at the next point in time. 

10 Figure 3 shows two traffic situations, according to. Figure 3a, 
no driver warning taking plac e occurs, and according to Figure 
3b, a driver warning taking place occurs. Future lane 202 
calculated for vehicle 200, which is determined, as shown 
above, taking into account possible corrections by the driver 

15 (e.g. fixed steering angle value), in Figure 3a shows no 
crossing with the edge marking shown as the dashed line, 
whereas in Figure 3b such a crossing is present. According to 
that, in Figure 3b the driver is warned at the time shown or a 
vehicle intervention is undertaken, since the driver will 

2 0 probably not hold the vehicle in the lane only by normal 
corrections, while in Figure 3a the warning does not take 
place occur because the driver will be able to do so. 

It is particularly advantageous if the The extent of the 
25 assumed correcting motion of the driver -rs may be a function 
of his state of attentiveness . In the case of a watchful 
driver, the correcting movement to be expected will lie, for 
example, at a steering angle of 10° and a transverse 
acceleration of 2 m/sec^, whereas this value is clearly lower 
30 in the case of an inattentive driver. The state of 

attentiveness of the driver is derived, for example, from 
steering movements, accelerator movements, body posture, etc, 
and then, as a function of the degree of attentiveness, the 
correcting values are determined, and these are incorporated 
35 in the calculation of the future track of the vehicle. 
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A second exemplary embodiment, which further improves the 
accuracy and reliability of the recognition of the existing 
situation, is shown in the flow diagram in Figure 4. This 
program, too, is run through at predefined time intervals. 
5 Analogously to the procedure method depicted before, here too, 
in step 300, a function is read in that indicates the pattern 
of the lane marking (right and/or left side) , which was 
derived from the image data of the image sensor system. In 
subsequent step 302, for first phase 4 (first time range, 

10 first distance range) one or more future tracks of the vehicle 
(e.g. track of the left vehicle edge and/or track of the right 
vehicle edge) are determined, for example, in the light of the 
vehicle speed and the steering angle and/or transverse 
acceleration, etc. In this phase, the assumption is made that 

15 there is no reaction by the driver, and consequently that no 
change in the steering angle or the transverse acceleration 
takes place occurs. It has become apparent that a time range 
of 0.5 sec is suitable as the duration of this first phase. In 
this context, the track is also represented using a functional 

2 0 equation. In one preferred implementation exemplary 

embodiment . in step 302, based on the ascertained function, 
predefined points of the track of the vehicle, which, for 
example lie at predetermined distances, are determined, to 
which the vehicle will move towards the end of the first 
25 phase. Corresponding track points (at corresponding distances) 
have been ascertained with respect to the edge marking region. 
Then the second phase begins, for whose duration a time of 0.1 
sec has been proven to be suitable. In this phase the driver 
reacts by turning the steering wheel away from the side 

3 0 marking. In this context, it is assumed that, at the actual 

time, a warning would have occurred. It is assumed that the 
steering wheel is turned by a certain value, or rather, that 
the vehicle travels a curve having a certain transverse 
acceleration. A 10° steering angle and 2 m/sec^ have shown 
35 themselves as suitable values. Consequently, in step 3 04, 
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starting from the end point of the track of the first phase, a 
track of the vehicle for the second phase is determined, an 
appropriate correction of the steering angle and/ or of the 
transverse acceleration being taken into consideration in the 
5 determination of the parameters of the function. After the end 
of the second phase, it is assumed that the vehicle will 
continue on at an unchanged steering angle, so that in one 
exemplary embodiment (not shown in Figure 4) an additional 
track calculation takes place occurs. In the second phase also 

10 and possibly the third phase too, in one preferred 

implementation exemplary embodiment , individual track points 
are calculated at certain distances. After determination of 
the future track curve of the vehicle, checking is done in 
step 3 06 as to whether the function of the pattern of the lane 

15 marking has crossings with the track curves of the first or 
second phase. This is done by evaluating the function 
equations or by comparison of the coordinates of corresponding 
track points. If this is not the case, then it is assumed that 
a situation is not present in which the vehicle's leaving the 

20 lane is a threat. However, if a crossing is detected, then 

according to step 3 08 an optical and/or acoustical warning is 
given to the driver and/or a vehicle intervention for the 
automatic correction of the steering is ascertained and 
carried out performed . The program is then ended and run 

25 through again at the next point in time. 

In an advantageous manner, — bot^i Both in this procedure method 
and the one depicted in the light of the first exemplary 
embodiment, the loudness of the warning, in the case of an 
30 acoustical warning, may be made variable and a function of the 
state of attentiveness of the driver. 

The effectiveness of the procedure method shown in Figure 4 is 
made clear in the light of the illustration in Figure 5. An 
35 x/y coordinate system is drawn in, whose centroid lies, for 
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example in the middle of the vehicle. In this context, vehicle 
400 travels in a traffic lane which is characterized by edge 
markings 402 and 404. Two tracks of the vehicle are also drawn 
in, one left and one right (406, 408) which describe the 
5 course of the left and the right vehicles edges . As was 

pointed out above, the course of the left and the right side 
markings (402, 404) relative to the vehicle are ascertained by 
a camera system. The course of the track markings is then 
shown with the aid of a mathematical equation, pr e f e rably 

10 e.g., a third order power function. In this way manner , the 
side markings are able to be calculated, for instance by the 
five points A, B, C, D, E for the left side markings, and by 
the points F, G, H, J, K for the right side markings. These 
points are selected so that, on the assumption of constant 

15 travel speed of the vehicle, they lie at certain time 

intervals ahead of the vehicle (e.g. A, F at 0 sec, B, G at 
0.25 sec, C, H at 0.5 sec, D, J at 0.75 sec and E, K at 1 
sec) . To determine the tracks of the vehicle, paths 406, 408 
are ascertained for the left and the right track of the 

20 vehicle. In this context, the travel speed and the current 
steering angle and/or the transverse acceleration of the 
vehicle make a difference. It is assumed that the vehicle 
travels at least in the next 0.5 sec using an unchanged 
steering angle (phase 1) . This corresponds to the area between 

25 points A and C, and F and H, respectively. The track of the 
vehicle is also represented by a mathematical equation 
corresponding to the representation of the pattern of the lane 
markings. In this way manner , within the extent of the first 
phase, points L, M, N may be determined for the left track and 

3 0 Q, R, S for the right track. In this context, the equations 
are set up in such a way manner that the left edge of the 
vehicle travels through the curve from L to N, and the right 
edge from Q to S. It is further assumed that the driver, if he 
were warned at the point in time shown, will, after ca 0.5 sec 

35 turn his steering wheel by a certain magnitude (e.g. ca 10°) in 
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the right direction, in order to avoid leaving the lane. In 
this way mariner , in phase 2 (area C to E and H to K, 
respectively, broken line area in phase 2) a new curve having 
a changed steering angle and/or a changed transverse 
5 acceleration is determined, which are determined by the points 
N, O and P and S, T and U, respectively. In this context, the 
track points of the vehicle tracks are calculated for the same 
distances as the track points of the lane markings, so that 
corresponding value pairs are formed. Thereafter, the patterns 

10 of the curves are compared (right track curve to right side 

edge curve, left track curve to left side marking curve) as to 
whether there are crossings or not. This tak e s place occurs in 
the implementation shown by comparison of the y coordinate of 
corresponding track points. Thus, a warning on the left side 

15 takes place occurs if either L lies above A, or M above B, or 
N above C, or O above D or P above E. Similarly, a warning on 
the right side takes place occurs if either Q lies below F, or 
R below G, or S below H, or t below J or U below K. 

2 0 As an alternative to this, the warning takes place occurs only 
if one of the points L to P lies above one of points A to E by 
a certain measure, as well as if one of points Q to U lies 
below one of points F to K by a certain measure. In this 
context, the value of the measure that was mentioned may also 

25 be negative. So that even an approach to the lane marking 

generates a warning. In this way manner, tolerances are taken 
into consideration. 

In this context, depending on the exemplary embodiment, the 
30 warning may take place occur on both vehicle sides or only on 
one vehicle side. 

In the exemplary embodiment just shown, the future track of 
the vehicle is calculated on the basis of a supposedly fixed 
35 steering of the driver. In practice it turns out that, in the 
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case of some drivers, relatively many false warnings may occur 
if such a driver swings back and forth between the lane 
markings . 



5 Therefore, in an additional advantageous exemplary embodiment 
the warning threshold for triggering the lane -leaving warning 
is dynamically changed by taking into consideration the degree 
of attentiveness of the driver when the calculation is made of 
the future track of the vehicle. The calculation of the future 

10 track of the vehicle, and therewith the warning threshold for 
triggering the warning becomes, in this context, a function of 
the behavior (performance characteristics) of the driver in 
the recent past . Depending on how much the vehicle travels 
curved tracks because of due to the steering movements of the 

15 driver, the calculation of the future track of the vehicle, 

and thereby also the warning threshold itself are modified. In 
the case of attentive drivers, this avoids many false 
warnings, but in the case of inattentive drivers, the warnings 
are triggered at the right time. 

20 

In general, in the above exemplary embodiment, a warning is 
triggered if a left warning boundary touches the left lane 
marking, or the right warning boundary touches the right lane 
marking. In this context, the warning boundary represents the 

25 future track of the vehicle, taking into consideration a 

possible estimated steering correction by the driver, if the 
latter drives closely towards the respective lane marking. The 
future track of the vehicle, and therewith the warning 
boundary are therefore curved. It has turned out that an 

30 inattentive driver, who is supposed to be warned by the 

system, as a rule makes very slight steering movements before 
leaving the lane. Therefore the vehicle travels a path having 
a relatively constant curvature. Now, — it is ess e ntial that, — in 
In the present exemplary embodiment, the warning boundaries be 

35 are brought in with respect to each other if the curvature of 
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the track that has been traveled by the vehicle has changed 
considerably (i.e. if comparatively strong steering movements 
have been executed by the driver) . This means that a greater 
curvature of the future track of the vehicle away from the 
5 lane marking is assumed. In response to a relatively constant 
curvature of the traveled track (and thus at very slight 
steering movements of the driver) the warning boundaries are 
broadened out, i.e. the possible future track of the vehicle 
is calculated using a lower change in the curvature. 

10 

In Figures 6 and 7 the vehicle is shown as 400. Furthermore, 
the left and right lane marking are shown as 402. 404 shows 
the future course of the vehicle without consideration of 
steering wheel changes, while the left and right warning 

15 boundaries are shown as 406 and 408, respectively. In this 

context, the left and right warning boundaries represent the 
future track of the vehicle while taking into consideration 
the correcting interventions of the driver. In this context. 
Figure 6 shows the situation in response to an attentive 

20 driver, while Figure 7 shows the situation in response to an 

inattentive driver. Consequently, in Figure 6 the curvature of 
the vehicle's course changed greatly in the past. It is for 
that reason that the warning boundaries run close together. In 
other words, the courses of the warning boundaries show a 

25 great deviation from the future course of the vehicle without 
steering correction. If the vehicle approaches the right or 
the left lane marking, a warning occurs only at a late time. 
In Figure 7 the situation is different. In the near past, the 
curvature of the course of the vehicle has hardly changed, and 

3 0 therefore the warning boundaries lie wide apart at a certain 
distance ahead of the vehicle. Thus, the courses of the 
warning boundaries here show a lesser deviation from the 
future course of the vehicle without steering correction. If 
the vehicle approaches closer to the lane marking, a warning 

35 is triggered in good time. 
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In the preferred exemplary enibodiment , this behavior is 
implemented in that, with the aid of steering movements, 
recorded, for example, from the signal of a steering angle 
sensor or a yaw rate sensor, a factor is determined using 
5 which the predefined correcting value is weighted. 

Consequently, when the calculation is made of the future track 
of the vehicle, and with that the course of the warning 
boundaries, a large correcting value is assumed in response to 
strong curvatures and steering movements, which leads to a 

10 greater change of the curvature of the future track, and with 
that, a drawing together of the warning boundaries, while in 
response to small steering movements in the past, a lower 
curvature factor is assumed, and consequently the warning 
boundaries lie far apart. In this context, the implementation 

15 takes place preferably occurs, as described in the case of the 
exemplary embodiments described above, within the scope of a 
computer program* 

The drawing together of the warning boundaries that was shown 

2 0 is not only undertaken as a function of the past of the 

vehicle's course, but, in one exemplary embodiment, 
alternatively or additionally even at time periods in which, 
for example, the measurement of the lane markings is 
uncertain. Therefore, in such situations a large factor is 
25 specified, which leads to a great curvature of the warning 

boundaries, and consequently to a drawing closer together of 
the warning boundaries. This avoids unnecessary signals as a 
response to uncertain recording of lane markings . 
In other exemplary embodiments, besides recognition of the 

3 0 attentiveness of the driver in the light of the curvature of 

the course of the vehicle in the past, or in the light of 
steering movements, a different recognition of attentiveness, 
such as one derived from accelerator movements, head posture, 
etc, is used. 
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Abstract 



ABSTRACT 



A method and a device ar e propos e d for driver information and 
for a reaction upon leaving the traffic lane by a vehicle. In 
this context, the course of at least one edge marking of the 
traffic lane is ascertained, the track of the vehicle that is 
to be expected is determined, and as a function of these 
variables the possible leaving of the traffic lane by the 
vehicle is ascertained. In the determination of the track of 
the vehicle, future reactions of the driver are taken into 
consideration . 
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